Conndence Based Dual Reinforcement Q-routing: an Adaptive Online Network Routing Algorithm

This paper describes and evaluates the Conndence-based Dual Reinforcement Q-Routing algorithm (CDRQ-Routing) for adap-tive packet routing in communication networks. CDRQ-Routing is based on an application of the Q-learning framework to network routing, as rst proposed by Littman and Boyan (1993). The main contribution of CDRQ-routing is an increased quantity and an improved quality of exploration. Compared to Q-Routing, the state-of-the-art adaptive Bellman-Ford Routing algorithm, and the non-adaptive shortest path method, CDRQ-Routing learns superior policies signiicantly faster. Moreover, the overhead due to exploration is shown to be insigniicant compared to the improvements achieved, which makes CDRQ-Routing a practical method for real communication networks.